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Abstract — In this paper, we adopt the Zeeman nonlinear heart model to discuss its stability 
and control its operation using emotional learning control (ELC). We also demonstrate the 
control of the heart model under threats of possible time delay introduced in the sensing 
loop. We compare the robustness of the ELC with other control methods such as the 
classical PID and the model predictive control (MPC) for the heart model under time delay 
attack. We have showed that ELC is more robust than the classical PID and the MPC. 

Index Terms — Emotional learning controller, TDS attack, nonlinear heartbeat model 

I. Introduction 

There are many design and analysis techniques developed for linear time-invariant (LTI) systems [1, 2]. 
However,_for nonlinear systems which could possibly be time- varying, one needs different methodology. A 
major part of nonlinear control theory studies how to extend the well-known methods for liner systems to 
nonlinear systems. 

One of the most complex but robust nonlinear systems is the human heart. Electrocardiogram (ECG) records 
potential differences between two electrodes located on the skin at predetermined positions on the chest to 
measure the electrical activities in cardiac tissue. A solitary cycle of ECG consists of the activities of 
relaxation and contraction of the heart (called the heart pumping actions). One can identify an ECG signal by 
its P, Q, R, S, and T peaks, PR and ST segments, PR and QT intervals, and QRX complex. Characteristic 
information took out from ECG can be used to assess cardiac health and identify potential heart problems. 
For example, important information extracted from ECG recording is the time between successive R-peaks 
which is stated to as an RR -interval. The changeability of the series of RR-intervals, known as heart rate 
variability (HRV), is being used to measure heart functions: identifying patients risk for a cardiovascular 
failure [3], as "an indicator for mortality following myocardial infraction" [4], and as a measure of the 
contacts between different control mechanisms of physiology like respiratory sinus arrhythmia. 
The development of mathematical models of heartbeat (ECG) with appropriate PQRST peaks, QRX 
complex, (PR, ST) segments, (PR, QT) intervals and HRV spectra has been and continue to be the subject of 
wide investigations with varying degrees of successes. A good model of ECG will make available a valuable 
tool for analyzing the various physiological conditions effects on the outlines of the ECG and for the 
assessment of diagnostic ECG signal processing devices. 

The form of ECG signal is the result of the propagation of electrical activities in myocardium and HRV is the 
result of physiological and neurological controls. In 1972 Zeeman presented a set of nonlinear dynamical 
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equations for heartbeat modeling [5-7] based on the Van der Pol-Lienard equation. These models are based 
on pace making generated by the Sino-Atrial (SA) node, which is the dominant pacemaker as compared with 
the slower one produced by the Atrio-Ventricular (AV) junction. Furthermore, these models did not take into 
consideration of sympathetic and parasympathetic modulations responsible for HRV generation [8]. 
In [9], the Zeeman's 2 nd -order ordinary differential equation (ODE) of the heartbeat model was modified by 
incorporating a switch (on/off) control variable demonstrating the pacemaker's mechanism of the 
contraction -relaxation for the heart. Whereas Jafarnia-Dabanloo et al in 2007 [10] modified the 3 rd -order 
nonlinear Zeeman model by adding control parameters that affect the frequency of the oscillation to control 
the HRV by using a neural network to produce the ECG signal. 

Yet another well-known approach to model the cardiac rhythms is based on the Van der Pol oscillators. In 
contrast to the Zeeman models, the coupled Van der Pol oscillator models at AV node making it have a more 
active role in pace making [11]. This model allows us to consider the effect of the coupling between the SA 
and the AV pacemakers in normal electrophysiological dynamics. 

In this paper, we study the stability of the heartbeat model that has been developed by Zeeman [5]. This 
model captures, at least qualitatively, three essential characteristics of cardiac dynamics: (i) a stable 
equilibrium, (ii) a threshold for triggering the action potential, and (iii) a return to equilibrium. The model 
consists of a 2 nd -order nonlinear ODE of the Lienard-type representing heartbeat dynamics [15]. 
In this era of pervasive communication technology, distant monitoring and follow-up of patients implanted 
with pacemakers (PMs) and implantable cardioverter-defibrillators (ICDs) is becoming very common. Most 
companies fit PMs and ICDs with wireless capability that communicates information to home transmitters 
then to physicians. These systems are widely used in the USA and are being introduced in Europe. Only 
recently, a tiny wireless pacemaker is launched in Europe by Nanostim [12]. The security of current and 
future biomedical devices is going to draw a major concern of public health in the future. Recently the U.S. 
Department of Homeland Security underscored some threats affecting almost 300 medical devices [13]. New 
revolution in wireless pacemakers [14] can further plan of attackers intending to access and sabotage 
biomedical devices. Attacks such as false data injection, denial of service and other types of attacks can make 
people's life who depend on these wireless enabled devices (e.g. pacemakers), difficult and dangerous. 
Therefore, it is reasonable to model the heartbeat pacemaker system under time-delay-switch (TDS) attacks 
and provide a possible solution. TDS attacks is a switched action "Off/Delay-by-x", where x specifies some 
random delay time, of the sensed system states or control signals of a system. We are going to model the 
TDS attack on the heartbeat control system (Fig.l) as a hybrid system. Then, we will provide a solution using 
emotional learning control to temper the effects of such an attack on the control of heartbeats. 
The first section of the paper introduces a common nonlinear model of heartbeat, i.e. Zeeman models. Also 
we discuss the stability of 2 nd -order heartbeat model using the indirect method. In the next section, we 
present the method of the emotional learning controller for control of the 2 nd -order heartbeat model. Note 
that, controllability and observability are assumed and have been studied and discussed elsewhere [15]. 
Based on nonlinear feedback Emotional Learning PI control technique and schematic method, Section 3 
organizes the control of the system. Finally, it presents experiments to verify that this method works well. 
The conclusion section shows that this method is powerful to track the ECG signal and is more robust 
compare to other control methods under TDS attack or random delay in the sensing loop. 




Figure 1 . Heart beat control model with possible control attack 
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II. Nonlinear Heartbeat Model 

Based on [6], the second-order nonlinear heartbeat model is given by: 



X x (t) 
X 2 (t) 



■-{X 3 x (t)-TX x (t) + X 2 (t)} 
s 



T>0 



(1) 



X l (t)-x d 

where the states X 1 (t) and X 2 (t) represent the length of a muscle fiber and a state related to 
electrochemical activities, x d indicates a typical muscle fiber length, s is a small positive constant which 
plays a role in fast eigenvalues of the system, and finally T shows tension in the muscle fiber. Parameter 
values are given in Table 1 . 

Table 1 Parameters value of model (1) 



Parameter 


x d 


T 


8 


value 


1.024 


1 


0.2 



Values in Table 1 have been checked by analyzing the equilibrium point stability using the well-known 
Lyapunov indirect stability theorem [16]. For this purpose, let A be the Jocobian matrix of Equation 1 at the 
origin. 
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(2) 



For the parameter x d = , the eigenvalues of A have the values of 3.618 and 1.382. Both are positive and 
show that the origin is not stable. 

The condition for the real part of the eigenvalue to be negative is 3X X - T > . So the system can be stable if 

X x > and X x < • These conditions can be satisfied if the value of x d be 1.024. For x d = 1.024 

, the stable equilibrium point is at (1.024,-0.0497) in the state space. In this case, all of the trajectories, 
irrespective to their primary conditions, go to the diastolic equilibrium point. 

The system stays at the stable equilibrium point endlessly, until the equilibrium point is stable, except there is 
an exterior excitation that forces the system move to a new equilibrium point. Based on this new stable 
system, a control input is added to the system (1) as shown below: 



X x (t) 
X 2 (t)_ 



-{X x 3 (t)-TX x (t) + X 2 (t)} 

8 



(3) 



_{X x {t)-x d ) + {x d -x s )u(t) 
where, when heart is in systolic state, x s is an additional parameter representing a typical fiber length and 
u(t) shows the cardiac pacemaker controls mechanism that leads heart into diastolic and systolic states. The 
feedback controller can be found in the form 

u(t) = -KX (4) 
and the new state after the attack (or random delay of feedback line) can be modeled by 

X = X(t-r) (5) 
In Equation 5, z is the time-delay and a positive integer. When time-delay is zero, the system works in its 
normal operation. The adversary can access to the communication line and delay the communication line. 
This model is true also for the small value of delay which can happen in the nature of the wireless sensor. 



87 



III. Emotional Learning Controller (ELC) 



Emotional learning controller was presented by Moren and Balkenious for the first time. They started to 
evolve computational models for parts of the human brain that carry out emotional functioning. In [17], a 
new computational model of brain emotional learning included amygdala, orbitofrontal cortex, and thalamus 
and finally sensory cortex has been presented, (Figure 2). 



Sensory 
Input (s) 



Thalamus 



Scnvorv Cortc\ 



Orbitofrontal Cor(c\ 




A in >u dulu 

Figure 2 Scheme of detail presented brain emotional learning model 

Amygdale is a small unit in the brain, which is involved in emotional evaluation of the stimuli. These 
emotional states and reactions derive attention signals and ultimately motor control commands. Some of 
inherent exciters are: hunger, pain, certain smells, etc. can excite the amygdale. The amygdale response to 
these stimulants is used in learning. On the other hand, the orbitofrontal cortex plays the role of a modifier of 
inappropriate responses and reactions of the amygdale. Numerous experiments on patients with damaged 
orbitofrontal-cortex have revealed that they are not able to adapt themselves to new conditions [18], in the 
other words, previous learning does not let them understand and response to new conditions. 
The ELC consists of two stages which carry out the learning and control process: one corresponds to 
information processed by the amygdala and the other by the orbitofrontal cortex. Basically, input stimuli 
(signals) are processed and used as a gain control of coefficients (synaptic efficacies) to affect future 
processing of stimuli and motor control commands. 

By ignoring some details of Figure 2, the computational emotional learning model of amygdale -orbitofrontal 
can be proposed in Figure 3. 



Sensory 



Input (si 




Figure 3 Overall computational model of ELC 

The output of the computational model can be found as: 

M =0 A -0 0C (6) 
where M is the model output, A and 0C are the amygdale and the orbitofrontal cortex unit output, 
respectively. 

The outputs of the amygdale and orbitofrontal units are described by 
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A =G A I s (7) 

°oc = G oc ' J s ( 8 ) 
where G A is the amygdale gain, I s is the sensory input, and G oc is the orbitofrontal cortex gain. The 
amygdale and orbitofrontal learning laws can be modeled as: 

AG oc =k r (0 M -I PR ) (9) 
AG A =k 2 -Max{0J PR -0 A ) (10) 

where I PR is the primary reward, k x and k 2 are rates of learning related to orbitofrontal and amygdale, 
respectively. The Equation (10) shows that the amygdale learning uses Max operation, therefore the 
amygdale gain is forced to have a consistently increasing deviation. This emphasizes the physiological fact of 
the amygdale unit which memorizes what it has been learned. 
Now, let us use Equations 6, 7 and 8 to find a general formula: 

M =(G A -G oc )I s (11) 
Based on Equation 11, we can conclude that the output for emotional learning system of the amygdale- 
orbitofrontal unit depends on the amygdale and orbitofrontal gains and the sensory inputs. It should be noted 
that these gains depend on the primary reward signal. 

In this paper, the sensory input of emotional learning system has been formulated by a format similar to self- 
tuning PID: 

/ (t) = K P e(t) + K D e(t) + K t fe(f ) dt (12) 

Jo 

where K P , K D and K t are the proportional gain, the derivative gain and the integral gain, respectively. The 
signal e(t) is the difference value between the tracking error and the output value at any time instant. 
However, to get a better response in front of noise, we used PI instead of PID as our sensory input to the 
amygdale-orbitofrontal emotional learning system. In the next section we will show that this controller 
improves the performance of the system in the attack operations and also track the reference ECG signal very 
well. 

IV. Results And Comparison 

We have simulated the 2 nd order heartbeat model to track the ECG signal using the Emotional Learning PI 
Controller (ELPIC). Then an external TDS attack or random feedback delay is applied to the model from a 
start time t s until a final time t F to show the performance of the ELPIC, which is compared with the classical 
PID and the model predictive controller (MPC). 

Figure 4 shows the ECG reference tracking X 2 (t) using the ELPIC. The line shows the output of the model 
controlled by the ELPIC and the points indicate the patient's ECG referenced signal [19]. The result shows 
that the ELPIC tracks the reference signal accurately. 

The TDS attack or feedback delay has been applied to the model from t s = 0.4 sec until t s = 0.45 sec with 
time-delay of r = 0.01s . For the purpose of visual comparison, the attack period and the amount of attack 
have been selected small because for larger period and time-delay value, the classical PI and MPC controllers 
goes out of visual bound. Figures 5 and 6 compare the ELPIC with the classical PI and the Matlab MPC. The 
results clearly show that ELPIC is more robust in front of TDS attack or unwanted random feedback delay. 

V. Conclusion 

In summary, we have shown that the ELC applied to Zeeman nonlinear heart model is able to track the ECG 
signal with high fidelity. Furthermore, the robustness of the ELC to control Zeeman heart model under time- 
delay switch attack was demonstrated. Its stability and control of its operation was evaluated. We have also 
showed that ELC is more robust than other common control schemes such as the classical PID and the model 
predictive control (MPC). 
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